Logging Standards

Logging is one of the core pillars of any successful product lifecycle. It gives complete traceability and empowers teams to trace and debug at a very low level to understand what's happening in the flow. This not only helps in troubleshooting but also helps at a great level to see if the flow is correct and it is expected to happen.

Especially when it comes to Microservice architecture style, logging plays a pivotal role in tracing the flow across services. Streaming the distributed logs to a centralized monitoring tool enables tracing the flow across services to make it a complete story.

Note: Spring boot by default comes with Slf4j logging feature inbuilt. There is no need to add any dependency to the project.

As seen in the above diagram, logs generated by each service are collected by log agent of any tool e.g. NeRelic which is deployed in each Node of the K8S cluster sends the logs to a centralized log store.

The logging related coding that needs to be done is covered in the section below. Each tool has it's own and it's installation depends on the vendor. Refer to specific vendor official documentation on how to install the agent at each node level.

Change	Class/Interface	Description
@Slf4j	import org.slf4j.Logger; import org.slf4j.LoggerFactory; @Slf4j Class PersonController { }	Class level annotation. Add this to every class and this will create a log object automatically which can be used further in anywhere in methods.
logger.trace("A TRACE Message”) logger.debug("A DEBUG Message"); logger.info("An INFO Message"); logger.warn("A WARN Message"); logger.error("An ERROR Message");	These are the logger statements that can be used anywhere in a program as per the requirement. Below section provides details on what level of logs to be used in what situation.
Application.properties/application.yaml	Any changes to logging configuration can be done in this file. Refer Spring boot documentation for the same. Its standard which is followed.

Change

Class/Interface

Description

@Slf4j

import org.slf4j.Logger;

import org.slf4j.LoggerFactory;

@Slf4j Class PersonController { }

Class level annotation. Add this to every class and this will create a log object automatically which can be used further in anywhere in methods.

logger.trace("A TRACE Message”)
logger.debug("A DEBUG Message");
logger.info("An INFO Message");
logger.warn("A WARN Message");
logger.error("An ERROR Message");

These are the logger statements that can be used anywhere in a program as per the requirement. Below section provides details on what level of logs to be used in what situation.

Application.properties/application.yaml

Any changes to logging configuration can be done in this file. Refer Spring boot documentation for the same. Its standard which is followed.

Logging Standards

The logging levels and log formats used in the Calibo Accelerate platform are as follows:

Logging Levels

The platform follows the standard logging levels to ensure appropriate categorization of log messages:
- DEBUG - Detailed information for debugging purposes.
- INFO - General operational information.
- ERROR - Error events that might still allow the application to continue running.
Log Formats

[timestamp] [logLevel] [traceId:spanId] [serviceName] message
- Timestamp - time when the log entry was generated.
- Log level - The severity of the log message.
- traceId - the unique identifier of the trace across microservices.
- ServiceName - the name of the service generating the log entry.
- message - the content of the log message.

Trace and Span IDs

Trace and span IDs are essential for distributed tracing. They allow us to correlate logs across multiple microservices, helping us follow the flow of a single request through the entire system.

Distributed Tracing

Trace Context Propagation

For distributed tracing, we propagate the trace and span IDs through appropriate headers. These IDs are used to connect related log entries across different services. The trace context is automatically propagated through HTTP headers, and the framework handles this propagation for us.

Error Handling

Happy Path Logging

In scenarios where operations proceed as expected, we follow a minimal logging approach. This ensures that log files are not cluttered with unnecessary information. We log at the INFO level for such cases.
Error Logging

In the event of errors or unexpected situations, we log at the appropriate severity level (WARN, ERROR, or FATAL) to provide detailed information for troubleshooting and debugging.

Examples:
- WARN: Unusual but not critical issues that do not prevent the system from functioning.
- ERROR: Errors that affect the operation of the system but do not cause it to crash.
- FATAL: Critical errors that lead to system crashes or data loss.

Best Practices

With the distributed logging and error handling practices enlisted below, the Calibo Accelerate platform aims to maintain a clear understanding of our system's behavior and health. Consistent and structured logging significantly aids in identifying and resolving issues promptly.

Follow consistent logging patterns and adhere to the defined log format.
Include trace and span IDs in log entries for traceability.
Use appropriate logging level to ensure that log files contain relevant information without excessive noise.
Log relevant context information for errors to aid in diagnosing issues.
Regularly review and update logging configurations to match the evolving project requirements.